In recent years, a medical observation system has been proposed, in which a point locking point is set independently from the setting of a focus position based on the measured data of the three dimensional shape of a portion to be observed (refer to PTL 1).
In addition, in a general surgical endoscopic observation, an endoscopist other than a surgeon moves the position of an endoscope in order to obtain a focused image of an observation target. Accordingly, the intentions of the surgeon and the operator of the endoscope sometimes do not match, and thus, there are cases where it is difficult to obtain a focused image of the observation target intended by the surgeon. A case where the surgeon directly operates the endoscope can be considered. However, in this case, the surgeon has to release the surgical instrument such as forceps held with both hands of the surgeon during operation, and thus, the surgical operation efficiency deteriorates.